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Abstract 


Being overconfident in one’ abilities is connected to poor self-judgment and it is essential to find ways 
how to reduce this phenomenon. The aim of the study was to research whether the act of taking a reading 
literacy test can be an effective means of reducing the overconfidence effect in the sample of Slovak 
teachers. Using purposive sampling method, a sample of 225 teachers from across school levels was 
obtained. The levels of overestimation (OE) and overplacement (OPE) effects were compared before 
and after having undergone the reading literacy test. The results showed that even just taking the test 
can reduce OE and to a lesser degree also OPE in all participants regardless of the feedback, which was 
in contrast to the original belief that just the experience itself does not affect overconfidence. Greater 
reduction in OE was observed in less competent participants, whereas none was observed among the 
more competent. These results showed that experience is also a form of debiasing. 
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Introduction 


Teachers have the task of passing on the knowledge to their students, but they can be 
overconfident and pass it on wrongly. The role of the teacher is very important for the development 
of the student, as his teaching activity influences the student's personality. However, if teachers 
have a non-objective self-assessment, there is an increased likelihood that the teacher will not 
be able to motivate his student to objective self-reflection. Many studies which have taken place 
over the last 40 years, have shown that people are overconfident (Boyd, 2014; Bracha & Brown, 
2010; Ghose, 2013; Gilovich et al., 2002; Kahneman & Tversky, 1979; Kahneman & Tversky, 
2004; Maratolli & Richardson, 1988; Tversky & Kahneman, 1974; Whitlestone, 2012). Many 
researchers such as Whitlestone (2012), Maratolli and Richardson (1998), Lichtenstein and 
Fischhoff (1977), Svenson (1981) and McGraw et al. (2004) have pointed out that not only 
laymen are overconfident, but experts too. Some research studies (Lichtenstein et al., 1982; 
Russo & Schoemaker, 1992) have even shown that experts often think of themselves as better 
than they really are and at the same time as better than their counterparts. 

Kearney and Sheffer (2015) have illustrated that college teachers are often convinced that 
their lectures are more comprehensible (overestimation effect) and that the performance they 
have given is better than that given by their colleagues (overplacement effect). Cross (1977) has 
documented superiority (overplacement effect) in college teachers as early as the 1980s, when 
he found that teachers thought they were better than their colleagues. 

Kruger and Dunning (1999) have pointed out that competent individuals tend to 
underestimate their abilities and performance, and incompetent individuals overestimate those. 
It seems that paying attention to the overconfidence effect is also important in professionals. 

This study reflects the view of Olsson (2014) that the overconfidence effect is formed 
by three constructs, and these represent the ways in which overconfidence effect is studied. 
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First construct is overplacement effect, which is about individuals overestimating their abilities 
and performance when compared to other people (Barbara & Odean, 2001; McGraw et al., 
2004; Shiller, 2005). Second component is overestimation effect, which pertains to individuals 
thinking that their abilities are better than they really are (Arkes et. al., 1981; Deaves et.al., 2008; 
Oskamp, 1965). The last construct is calibration of probabilities, which is about individuals 
overestimating their predictions (Hoorens, 1995; Svenson, 1981). 

In addition, this study considers the research of Kruger and Dunning (1999) pointing 
to the role of competence in relation to the overestimation and underestimation of abilities. 
McKenzie (1997) found that a decrease in overconfidence also occurs when people are forced 
to consider alternative outcomes. Koriat et.al (1980) has shown that excessive self-confidence 
can be reduced even when participants are asked to compile a list of counterarguments to their 
estimates before writing their final estimate. Kruger and Dunning (1999) have pointed out 
that experience is not a debiasing method, whereas Krueger and Mueller (2002) have argued 
that experience can also affect one's self-evaluation if the individual is prepared for it. So, the 
question remains whether it is possible to reduce the overconfidence effect only by experience 
alone? 


Research Problem 


Overconfidence is connected to poor self-judgment and finding ways of affecting it are 
of importance among teaching professionals. It was originally believed that experience does not 
calibrate the perception of oneself and one’s performance (Kruger & Dunning, 1999), but later 
it was found that feedback can influence self-evaluation of participants, if it is given well (Krajé 
& Ortmann, 2008). This study aimed to answer the question whether experience with the task 
can be an effective means of reducing overconfidence in teachers as it is also a sort of feedback, 
and can serve as a form of debiasing. Specifically, how does the experience with Reading 
Literacy Test (RLT) affect the overestimation and overplacement effects among teachers? 


Research Focus 


Stanovich (2011) was convinced that debiasing can occur even if an individual is able 
to suppress his primary automatic responses. At the same time, Stanovich (2011) added that 
in order for debiasing to take place, it is necessary for the individual to have space in the 
cognitive capacity that would allow him to better detect distortion and subsequently find a 
solution. Larrick (1993) argued that we can look at debiasing in two ways. The first of them is 
characterized by the fact that it focuses on the decision-maker himself, so that the level of his 
knowledge base increases, for instance through education, or by providing him with tools to 
streamline his decision-making process. The second approach is different in that it focuses on 
the environment in which the individual makes decisions by creating optimal conditions in the 
environment that positively affect the decision-making process. This research aims to verify 
the notion of Larrick (1993) whether experience with RLT does increase the level of learning 
in the individual, and the notion of Kruger and Dunning (1999) whether it would correct the 
underestimating or overestimating tendencies. 


Research Aim and Research Questions 
The aim of the research was to study whether the experience with RLT affects 
overestimation and overplacement effects, and if the degree of competence plays a role in this 


process. The background of the study was research by Kruger and Dunning (1999), who pointed 
out that if a competent individual receives feedback, his perception becomes closer to reality, 
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but if an incompetent individual receives objective feedback, his perception does not change. 
Ramaprasad (1983) argued that in order for a person to know a blind spot in his perception, 
it is important to receive feedback that would inform him of the very difference between the 
expected and actual level of his abilities or performance. Nordquist (2014) added that even 
through feedback, an individual can gain knowledge that his abilities and competencies may 
not really be what he considers them to be. Is an experience also a form of feedback? If so, how 
does it affect the less competent and how more competent? 


Following research questions were formulated: 


RQ; Does the experience with RLT affect OE? 

RQ,; Does the experience with RLT affect OPE? 

RO; Does the effect of the RLT experience on the OE depend on the level of competence? 
RQ, Does the effect of the RLT experience on the OPE depend on the level of competence? 


Research Methodology 
General Background 


This study was part of the larger research project APVV-0361-12 titled “Rozhodovanie 
profesionalov: Procesualne, osobnostné a socialne aspekty [Decision making of professionals: 
Processual, personality, and social aspects]” taking place at the Slovak Academy of Sciences, 
which focused on different ways of reducing the overconfidence effect. To be able to answer the 
research question about causal relationship, experimental methodology with within-subjects 
research design was chosen, and the study was conducted using online platform Qualtrics.com. 
The participants were asked to estimate their skills and performance prior and after having 
completed the RLT. 


Sample 


Purposive sampling method was used, and 14 Slovak schools were contacted to participate 
in the study. Those schools were found in the database of National Institute for Certified 
Educational Measurement and were contacted based on the previous research cooperation. 
The goal was to recruit as many participants as possible to achieve sufficient statistical power. 
Eventually, total of nine schools agreed to partake in the research and the final sample consisted 
of 223 teachers, which is the best possible result as research participants are notoriously hard 
to find. The mean age of teachers was 45.48 (SD = 10.13) and the age range was 24-69 years. 
Teachers came from the variety of school levels: primary (1 = 38), lower-secondary (n = 82), 
upper-secondary (n = 95), and universities (” = 8). Participation of the research was voluntary, 
and teachers filled out the test battery in the presence of the researcher at school, while some 
individual participants partook the experiment from home. 


Instrument and Procedures 
Reading Literacy Test (RLT) 
The test was created by Mgr. Branislav Hudcovsky from the National Institute for Certified 
Educational Measurements (NUCEM) in Slovakia for the purpose of this research based on 


the pilot testing on the population of high school seniors, which NUCEM regularly organizes 
(Krause & Kurincova Cavojova, 2018). The test consists of three stories which participants 
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ought to read and then answer 10 questions pertaining to the texts (30 questions total). Only first 
of the three stories was used to test for the effect of experience. The psychometric properties of 
the test are not known; however, those tests are used in regular testing of high school students. 
For the purpose of this research, it is, therefore, less important whether it actually measures 
what it is supposed to measure, because it is used in practical settings, nonetheless. 


Overestimation effect (OE) 


Overestimation effect was measured by asking participants to estimate the number of 
correct answers on the RLT ona scale from 0 to 10. 


Overplacement effect (OPE) 


Overplacement was measured by asking participants to estimate their placement among 
other participants on a scale from | to 100 where | is the best and 100 is the worst placement. 


Research procedure 


Participants first answered demographic questions about their age, gender and at what 
school level they were teaching and continued to the questions on OE and OPE, which were 
used based on the previous research in the field that uses two questions to measure these two 
effects (Olsson 2014). Afterwards, they moved on to completing the Reading Literacy Test 
and questions related to what the participants read. Finally, after reading and then answering 
all the questions, the participants again filled in the questions on OE and OPE. Based on the 
recommendations of Olsson (2014), OE was measured as comparison between expected 
performance and actual one, and OPE as self-evaluation of one’s performance relative to other 
people. 


Data Analysis 
The statistical methods used to analyse data are listed in Table 1. The data were analysed 


using Statistical Package for the Social Sciences (SPSS), version 20. 


Table 1 
Methods of Answering the Research Questions 


Related t-test with OE before and after RLT as dependent 
RQ,: Does the experience with RLT affect OE? variables. Cohen's d was computed to compare the effect 
sizes of RLT on reducing OE and OPE. 





Related t-test with OPE before and after RLT as dependent 
RQ,: Does the experience with RLT affect OPE? variables. Cohen's d was computed to compare the effect 
sizes of RLT on reducing OE and OPE. 





RQ,: Does the impact of the RLT experience on the _ Related t-tests with OE and OPE before and after RLT 


OE depend on the level of competence? as dependent variables in groups of participants with 
RQ,: Does the impact of the RLT experience on the _ different levels of competence with cut-offs on 33° and 66" 
OPE depend on the level of competence? percentiles. 
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Research Results 
Effect of Experience with RLT on OE and OPE 


To check whether the actual experience with the Reading Literacy Test will serve as a form 
of feedback and will affect overplacement and overestimation (RQ,,RQ,), the overplacement 
and overestimation indices before and after taking RLT were compared using t-test for related 
scores. The results showed that taking RLT significantly reduced overestimation (M, = 1.21, 
SD, = 2.31 vs. M, = 0.13, SD, = 2.02), t (222) = 8.41, p <.001, d= .80. The results also show 
that the experience with the Reading Literacy Test decreased overplacement (M, = 5.32, SD, 
= 33.03 vs M, = 1.92, SD, = 33.32), ¢ (222) = 2.45, p = .015, d = .23), which was a small 
difference. Comparison of effect sizes indicates that experience affects overestimation to the 
greater degree than overplacement. 


Effect of Experience with RLT on OE and OPE in Relation to Competence 


To check whether differences in pre- and post-test OE and OPE were differently 
pronounced based on the competence levels (RQ,, RQ,), individual related t-tests were run in 
the groups of participants with different levels of competence based on the cut-offs on 33" and 
66" percentiles. 

Experience with RLT in less competent participants significantly reduced OE (M7, = 2.36, 
SD ,= 1.85 vs M,= 0.79, SD, = 1.92), t (142) = 10.67, p <.001, d= 1.27, which was a big effect. 
The results also indicated that the experience with RLT in less competent participants did not 
affect OPE (p >.05). At the same time, the analysis of more competent participants showed that 
the experience with RLT did not affect OE or OPE (p >.05). More competent, regardless of their 
experience with RLT, tended to underestimate themselves. 


Comparison Differences in OE and OPE among Differently Competent Individuals 


As part of the analyses, less and more competent participants were compared in the OE 
and OPE using independent-samples t-test. In comparing all participants, it was found that 
less competent participants (M = 2.36, SD = 1.85) were significantly more overestimated (OE) 
than more competent (= -0.84, SD = 1.50), t (221) = 14.02, p <.001, d= 1.33. Similarly, less 
competent participants (MW = 20.32, SD = 27.3) showed significantly greater overplacement 
effect in the first measurement than the more competent (M = -21.48, SD = 24.28), ¢ (221) = 
11.40, p <.001, d= 1.08. 


Discussion 


The aim of the study was to see whether the experience with RLT would affect the 
overestimation and overplacement effects and to examine whether the degree of competence 
played a role in this process. Results show that experience with RLT influenced OE and OPE 
in the whole research sample with a very large effect size. Competence played a significant 
role with less competent participants being overestimated before completing the RLT and more 
competent participants underestimated. However, after the experience with the test, competence 
did not play a role in OE, as experience with RLT inhibited OE in all participants, with the 
difference that in less competent it led to a more realistic estimate (although still overestimated) 
and in more competent to even more underestimating. 

In the experience with RLT among the more competent people, it was found that more 
competent participants underestimated themselves even more after the experience with RLT 
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than before completing the RLT. This can potentially be explained by Hilton et al. (2011) 
who pointed that the complexity of the presented tasks also played an important role in the 
overconfidence effect. They wrote that for complex tasks individuals rated their estimates as 
below average when compared to other people. And RLT as such can be viewed as a complex 
task. 

Bazerman (2002) showed that providing individual with information about his cognitive 
biases can be a sufficient form of debiasing. In this study, the participants were able to gain 
not only a cognitive insight, but also own emotional experience from the RLT, which is an 
important part in the reduction of the overconfidence effect (Bazerman, 2002). The results of 
this study are also aligned with Stanovich (2011), who claimed that debiasing can occur when 
an individual is able to suppress his primary automatic responses, and by the use of conscious 
processes find mental cues which lead to the elimination of the present cognitive distortion. 
Ramaprasad (1983) writes that in order for the individual to be aware of a blind spot in one’s 
self-perception, it is important that he be given feedback about the discrepancies between the 
expected and actual performance. The present study had experience with RLT as such feedback 
as participants could compare the expected performance with the actual one. 

Pintrich and Schunk (2002) discuss that individuals who are actively involved in 
receiving feedback show greater self-efficacy afterwards, which motivates them to perform 
better in the future. In the case of this study, participants were active agents of feedback, as 
they were not given it from the environment, but based on their own experience with RLT. The 
study by Wiliam (2011) showed that concrete feedback leads to great improvement in the future 
testing, as compared to the feedback given on a point-scale. In this study, the feedback was also 
concrete, which the participants addressed to themselves based on their own experience. 

It is worth considering the findings of the meta-analysis by Smither et al. (2005) who 
discovered that in order for the feedback to be the most effective, it is essential that the message 
suggests that change is necessary, but also that the individual is open to change and perceives 
an inner desire to change his behaviour while believing that change is possible. Feedback can be 
of different valences, which play a role in the adjustment of thoughts and behaviour (Jaworski 
& Kohli, 1991). It was the valence of the feedback, which was the core of by Krajé and Ortman 
(2008) criticism of the original research by Kruger and Dunning (1999) by showing that the 
incompetent do not adjust their estimates because of the metacognitive deficit, as was believed 
by Kruger and Dunning (1999), but because the feedback they were given was negative for 
them. 

Kruger and Dunning (1999) originally wrote that if individuals are confronted with 
objective results, they do not adjust their abilities and performance. This study shows that such 
statement is only partially true. The results of this study show that even the very experience that 
participants have allows them to see themselves better in the context of their performance, even 
if they are among the less competent group. However, ignorance of the performance of others 
makes them, despite their initial estimate, still think they are better off than other people. In 
other words, it is not so much about their metacognitive deficit as Kruger and Dunning (1999) 
believe, which prevents them from seeing their level of incompetence, but rather ignorance of 
other people's performance. 

However, this may also explain what Burson (2006), Krueger and Mueller (2002) and 
Yarkoni (2010) argued that even if incompetent individuals are overestimated, they do not rank 
higher than competent ones. In other words, an incompetent person does not think he is smarter 
than a more competent one. As part of future research, less competent participants should be 
confronted with other people's objective results to discern whether they would continue to 
believe that they are better off than they actually are, despite having an objective knowledge of 
other people's performance. 
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Conclusions and Implications 


The present study supports the original Kruger and Dunning’s findings by showing 
that less competent participants believe they are better off than others and have the tendency 
to overestimate their performance (OPE), while the more competence have the tendency 
to underestimate. On the other hand, this study disproves their claim that it is due to the 
metacognitive deficit, which prevents individuals in adjusting their self-perception, as this study 
showed that even own experience can lead to the change in perception of one’s performance 
(OE) in the less competent participants. 

Based on the results of this study, the criticism of original Kruger and Dunning’s study 
on the basis of not considering the statistical explanation of the wrong self-evaluation and 
not researching the differences in metacognition between more and less competent, seems 
justified. The inaccuracy in perception is not always needed to be explained by the deficit in 
metacognition, but also as combination of universal cognitive distortions, such as “better than 
average” or “regression to the mean.” 

The study showed that own test experience can also be type of debiasing. It seems 
that just taking the RLT affected the overconfidence, as the participants largely overestimated 
themselves before the test, but after the experience with the test their overestimation decreased. 
It was found that the experience with RLT in less competent participants significantly reduced 
overestimation effect. The results of this study also show that one's own experience can change 
perception of oneself and one’s abilities, even if one is less competent when compared to other 
people. The results of this study can be summarized in such a way that even having an experience 
can change an individual's perception, even if the individual is less competent. Therefore, it is 
important to give people the opportunity to adjust their own perceptions, for example in the 
form of experience, as it was shown to also be a form of debiasing. 

Regarding teachers, one way to provide them with experience is to let them take the tests 
they are teaching for. This study used a Reading Literacy Test designed for high school seniors 
and showed that teachers can be overconfident in their ability on this test. It is suggested that 
teaching professionals are given the opportunity to calibrate their believed skills with their 
actual performance, as this can reduce their overconfidence. 
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